Automated or autonomous vehicles (AVs) can process environmental data in order to control operation of the on-board systems of the AV, such as the acceleration, braking, and steering systems. In certain situations (e.g., to perform corrective maneuvering), a user may assertively, or via AV request, take over manual control of the AV's on-board systems.